The field of the present invention relates in general to a system and method for automatically tracking, measuring, and facilitating the progress of software development in a digital computer system. In particular, the present invention relates-generally to software development management systems; particularly Gantt chart software, defect tracking systems, and debuggers by providing a code based software development management and automated debugging system.
The management of software development is a new and difficult discipline. The inadequacy of software management techniques is demonstrated by repeated studies showing that at least seventy percent of all software projects are never completed. Without detailed, accurate measurement, runaway development costs, particularly in large organizations, are difficult to detect and stop. To track all the aspects of a project accurately, software development managers need an ongoing awareness of each developer""s activities, the option to transfer sections of code easily from one developer to another, and measurable criteria on which to base shipping decisions. The development process still costs at least one hundred dollars per line of code, with a typical software project containing thousands of lines of code; As development progresses, costs go up as a manager""s ability to effectively manage the project goes down.
Most managers rely on Gantt charts to track a project""s progress. However, Gantt charts require regular manual updating in order to accurately reflect the progress of the project. Managers, when under pressure to complete software, neglect to update the Gantt chart, making the Gantt chart irrelevant. Even if Gantt charts are updated regularly, they are of limited use because they are based on what people say: genuine complexity and technical-jargon make code overwhelmingly intimidating for anyone but its author to understand. Consequently, one of the few aspects of a modem large business enterprise that cannot be audited is software development.
Further, every time code is changed (xe2x80x9cversionedxe2x80x9d), its behavior will change in a way that cannot be predicted. Most of these changes produce intended and beneficial behavior, but some produce neither beneficial nor intended behavior. Erroneous, detrimental or deleterious behaviors are common called bugs. Bugs, which are introduced by new versions, are often not detected until long after they are introduced. Bugs remain undetected because most users have neither the time nor the tools to run regression tests, which check that working code remains working after it""s been changed.
Automated version control systems have been an integral part of software engineering for many years. They attempt to address the above noted, intrinsic problems of code development by essentially providing a data store for versions of code. Prior art enhancements to version management do notify managers about code changes, but don""t teach how to use version derived information to update Gantt charts.
The central underlying problem is the developer""s monopoly on information. If two developers (A and B) have the same level of skill, developer A cannot easily understand B""s code and vice-versa. Therefore, only A can effectively work on A""s code. This is what will hereafter be referred to as the xe2x80x9cdeveloper""s monopoly on information.xe2x80x9d Previous enhancements to version control systems have attempted to break the developer""s monopoly on information by forcing developers to enter reasons for entry and/or linked this with xe2x80x9cdiffsxe2x80x9d (versions with textual differences highlighted) of the code, but these systems are limited by forcing the developer to manually explain the reasons for the changes. Further, these enhancements don""t teach how to derive the developer""s intent directly from the code.
Problems of the Individual Developer
Code is so complicated that it is extremely difficult for an individual engineer to maintain quality as it evolves. Bugs, new and existing, are one of the most unavoidable, annoying, and time-consuming problems in the development of code. Each bug is unique, which is why there have been no successful automated debugging tools. However, bugs can be grouped into general classes:
Crashes: Crashing is the most severe, and one of the most common, errors. Developers often don""t know what caused a crash, so cannot reproduce and fix the problem. In addition, developers will sometimes leave crashes in a program in the rush to ship code.
Endless Loops: Developers often accidentally write endless loops. When a program hits an endless loop, it continues to run/loop until the program (or the computer) is shut off. Endless loops are particularly time consuming because the developer doesn""t know the cause, and may assume the program is just being slow instead of endlessly looping; Therefore, discovering endless loops takes a great deal of time.
A Statement Is Not Executed When It Should Be: One of the most common types of discrepancies is that a statement (line of code) will not be executed under conditions where the developer intended that it should. Sometimes these bugs are simple to solve, but, when they aren""t, their causes can take hours to find.
Bad Data: An even more time consuming-type of bug is finding out why a particular number or other piece of data is not what it should be. Sometimes the reason is simple, but often the number that shows up incorrect is the result of a calculation that occurred incorrectly somewhere else in the program. This incorrect calculation causes a chain reaction that manifests itself in the bad data that was originally detected. The developer must laboriously track this chain of data back to its source.
Something That Used To Work No Longer Does: Developers frequently make changes in code that breaks previously working code; This often happens when a developer is editing the code to address a problem in one area and does not anticipate the effects of the edit on other areas of functionality. Regression tests, well described in prior art, are an attempt to catch these types of inadvertently created bugs. However, regression tests only provide pass/fail information and do not pinpoint the edit that caused the break-. Further, the demands of development consume the time that developers might use to go back and test thoroughly. While quality assurance organizations have banks of regression tests that are automatically run, these require releasing the code to QA: so, they aren""t run on the code while it""s being developed.
The developer""s information monopoly applies not only to managers but also to other developers. This monopoly makes it hard to take over another developer""s code, especially-when the original developer is not available to explain the code. Most developers taking over the project will look at the code, and say it needs a rewrite (because they write differently). This rewriting is likely to introduce new bugs, making the program less solid than it was when transfer occurred.
Code transfer is a common, and expensive, software management problem. While many academic and commercial attempts to solve this problem have been tried, no successful automated system for code transfer has been developed.
Application Ser. No. 08/955,028 filed Oct. 10, 1997 (now abandoned) by the assignee herein provided beneficial features of xe2x80x9cautomatic bugsxe2x80x9d and is incorporated herein by reference. Application Ser. No. 08/955,835 file Oct. 10, 1997 (now abandoned) by the assignee herein provided beneficial features of xe2x80x9cthe monitor processxe2x80x9d and is incorporated herein by reference. Application Ser. No. 08/955,163 file Oct. 10, 1997 (now abandoned) by the assignee herein provided beneficial features of xe2x80x9cImplicit Ul Testxe2x80x9d and is incorporated herein by reference. Application Ser. No. 08/956,651 file Oct. 10, 1997 (now abandoned) by the assignee herein provided beneficial features of xe2x80x9ctransfer to another developerxe2x80x9d and is incorporated herein by reference. Application Ser. No. 08/954,935 file Oct. 10, 1997 (now abandoned) by the assignee herein provided beneficial features of xe2x80x9crevert until broken bugxe2x80x9d and is incorporated herein by reference.
Living Gantt chart expands project management software based on the Gantt chart. FIG. 1 shows a block diagram of a data processing system for software development, which may be utilized to implement the present invention. Using the Gantt chart editor (Microsoft Project in exemplary embodiment) 5 on his computer 2, the manager 1 accesses the Modified Planning Document 7. The invention 6 accesses the modified planning document by extending the user interface of the Gantt chart editor. The invention stores its representation of Gantt chart in the data store 8. While the block diagram shows that the data store is accessible over the network, it can also be local to the manager""s computer.
Each task line on the Living Gantt chart can be xe2x80x9clinkedxe2x80x9d to a particular module, procedure, crash, test or other items that the invention extracts from the structure and/or behavior of the target code. The manager can link these code-generated items with non-specific human generated tasks on the Gantt chart. Once this link is established, the invention automatically maintains it. Neither the developer nor the manager needs to manually maintain the Living Gantt chart; it updates itself when the manager opens the Gantt chart. Living Gantt chart also allows senior management and other nontechnical team members to view a project""s progress. This information allows them to be able to plan accurately and to coordinate software development activities with marketing campaigns, sales forecasts, and revenue projections.
The most fundamental capability of Living Gantt chart is to create xe2x80x9cplanning documentsxe2x80x9d (Gantt charts in the exemplary environment) which consist of abstract xe2x80x9ctasksxe2x80x9d from code. The code under development is called the xe2x80x9ctarget codexe2x80x9d, which is being developed by xe2x80x9cdeveloper(s)xe2x80x9d. The xe2x80x9cmanagerxe2x80x9d informs the invention of the target""s code location. The invention parses the target code into modules and procedures. The invention then creates a hierarchically organized planning document. The invention turns code into Gantt charts. Gantt charts derived from code are called xe2x80x9cderived planning documents.xe2x80x9d
The structure of the derived planning document and the target code are stored-in a persistent xe2x80x9cdata storexe2x80x9d. The invention maintains versions of the target code in the data store. Versions are maintained for the target code such that versions of modules and procedures are retrievable and comparable. Using the versions of target code stored in the data store, the invention tracks the development in the derived planning document, which is now referred to as the xe2x80x9cmodified planning documentxe2x80x9d. The invention extends the capabilities of the modified planning document by letting the developer match items in existing planning documents with selected parts of the target code. For example, a procedure p in module m might be linked to a Gantt chart task xe2x80x9cdevelop customer service formxe2x80x9d.
The invention uses the linkage between xe2x80x9ctasksxe2x80x9d and elements of the target code to update the modified planning document. The invention compares individual linked target code elements to determine where progress is actually being made, and points which are not advancing. Active code is easily identified as the code most recently edged.
The invention allows managers to see dusty projects (those that have not been touched in a designated period of time) or xe2x80x9cuntouchedxe2x80x9d projects (those that have never been worked on at all).
The invention compares versions of procedures to show when the engineer makes repeated and/or frequent edits. If the developer keeps cycling between edits over a period of days, it identifies the few lines of code that management is spending the most resources on and that the engineer is least able to address. Tasks linked to xe2x80x9cstuckxe2x80x9d target code elements are marked on the modified planning document. If linked tasks are bound to new code elements, that code is marked new.
Almost all bugs fall in one of four categories:
Crash Bugxe2x80x94When run, the target code generates an unhandled exception (xe2x80x9ccrashesxe2x80x9d).
Loop Bugxe2x80x94The code waits forever (xe2x80x9changsxe2x80x9d).
Control Flow Bugxe2x80x94The code does the wrong thing (xe2x80x9ccontrol flowxe2x80x9d); that is, the code does not execute the statements the developer expects.
Data Bugxe2x80x94The code runs correctly, but one or more the target code""s variables or outputs are set to a value that developer doesn""t expect.
The invention provides mechanisms for automating finding the causes of and suggesting solutions of these four types of bugs.
The invention needs to detect crashes when they happen, which is done by the xe2x80x9cmonitor processxe2x80x9d. In addition to providing a context for the invention, FIG. 1 also shows how a developer 3 uses the invention as well as how the invention interacts with the other elements of the developer""s computer 4. The developer edits the target code 11 via the development environment (Microsoft Visual Basic in the exemplary embodiment) 9. The invention""s monitor process 10 extends the development environment, which detects the developer""s actions. The invention""s instrumented nerves 12 provide information from the running program to the monitor process.
When the monitor process detects that the target code crashes, it determines the element (module, procedure, and statement) of the target code that the operating system""s exception handler provides. These entries provide a key into the data store for an object referred to as a xe2x80x9ccrash-bug.xe2x80x9d Each time the target code crashes; the circumstances of the crash (date, time, user, computer name, etc.) are added-to the data store.
Since exceptions codes for any particular operating system and programming language are well defined, the cause and most common solutions to crashes can be determined. The invention maintains a set of xe2x80x9clanguage specific analyzersxe2x80x9d for a subset of the exceptions generated by the development environment and/or operating system.
When the invention""s monitor process detects a crash, the statement that caused the crash is analyzed by the language specific analyzer, for the specific exception. The language specific analyzer parses the statement to 1) generate a hypothesis about the cause of the crash, 2) suggest code to prevent the crash recurring the next time this statement is reached, and/or 3) present a strategy for investigating indirect causes of the exception.
By suggesting analysis and solutions to crashes when they happen, the developer also benefits from the target code in being able to solve problems faster. In addition, the language specific analyzers provide a mechanism for applying the collective knowledge of developers, not just in documentation, but also on the crash as it is happening.
Information about the crash bug is available to the manager browsing the modified planning document, by accessing details of the tasks linked to automatic crash bugs. The invention provides forms in the Gantt chart editor that let the manager retrieve the circumstances of the crash, the target code itself, and how the developer decided to respond to the crash. The ability to view the areas of the target code as well as a history of its run-time behavior is referred to as xe2x80x9cdrilling down to the code.xe2x80x9d
When the target code crashes, the developer is often left in the unenviable position of being functionally unable to repeat the failure. Without the ability to reproduce the events leading up to the crash, the developer is severely handicapped in any attempts to understand, let alone correct the program flaw.
The monitor process records the sequence of the user""s Ul actions. Note that monitor process may or may not incorporate third-party tools for recording Ul tests, the implicit nature of the test is what is not specified in prior art. The invention assumes that every run might end in a crash, and begins to record actions as soon as it detects that the developer has started the target code. If the target code crashes, the sequence of actions is placed in the data store and labeled as an xe2x80x9cImplicit Ul Test.xe2x80x9d
Since Implicit Ul tests are associated with each crash, they are also available for display in the modified planning document, either as individual Hems and/or as child tasks to crash bugs.
Effectively endless loops are a time consuming class of bug. Unfortunately, they are a provably intractable annoyance: the body of computer science theory includes proofs that there is no way to predict when any arbitrary loop will exit, or, in fact, if it will exit at all. A loop that has iterated several million times may exit on the next pass. It is often impossible to tell for sure. However, the distribution of behavior among loops in actual code affords a heuristic loophole that the exemplary embodiment capitalizes on: most loops terminate relatively early on. Thus, the exemplary embodiment uses nerves to keep track of how many times a given loop has iterated since it was entered, and halts execution if the number of iterations passes a threshold. Since the threshold is predetermined by the developer, and the behavior may be disabled or customized on a loop by loop basis, this is a reasonable approximation to effective xe2x80x98infinitexe2x80x99 loop detection.
To intercept endless loops, the invention adds the step of instrumenting the code with a set of function calls termed xe2x80x9cnervesxe2x80x9d. Instrumenting code is the insertion of statements that are placed there solely to monitor the run-time behavior.
Inserting the C/C++ assert statement is one of the most common types of instrumention. Several other inventions have taught instrumenting code for finding memory leaks, but xe2x80x9cnervesxe2x80x9d are used for control flow debugging. Loop Bug uses the initLoop, checkLoop, and endLoop nerves.
The invention instruments each loop in the target code by putting an xe2x80x9cinitLoopxe2x80x9d nerve before loop entry, a xe2x80x9ccheckLoopxe2x80x9d nerve at the loop iterator, and an xe2x80x9cendLoopxe2x80x9d nerve at each loop exit. Each time the loop iterates the checkLoop nerve is called. The checkLoop nerve maintains a count of how many times a xe2x80x9ctarget loopxe2x80x9d executes. If the count exceeds a threshold, the invention interrupts the target code""s execution.
It then reports to the developer that it is suspects that the loop is endless and displays a set of options for 1) rewriting the code so the loop will not be endless, 2) raising the threshold, 3) marking the loop as one which is intentionally endless (e.g. a message handling loop). The loop bug information includes the loop""s location in the code, the iteration count, when the loop was written and the action taken by the developer. Loop bugs are reflected in the modified planning document in a manner similar to crash bugs.
The invention""s method for detecting loops is further extended to finding why the target code follows unexpected control flow paths. The invention expresses this with a control flow bug. The developer selects a statement of the target code that should be called when the target code is executed.
The invention instruments the target code with traceEnter (at procedure entry), traceExit (at procedure exit) and traceBranch (for if/then/else and switch statements). Loops are already instrumented as described above. CheckLoop nerves function as traceBranch nerves when control flow bugs are being run. The developer then executes the target code. The nerves compile a record of the target code""s execution, called the xe2x80x9cproximal control tracexe2x80x9d. The invention uses the proximal control trace to report to the developer if the procedure is even called, or failing this if any procedure calling the procedure is called.
If the target code has entered the procedure, the invention determines how xe2x80x9cclosexe2x80x9d the execution got to the target statement. xe2x80x9cClosenessxe2x80x9d is measured by the number of statements between the procedure entry point and the target statement. The invention then identifies the branch statements (including loop statements). From this information the invention then generates a hypothesis that the target statement would have been called had the intervening branches been executed. The expression that prevented the statement from being called is the immediate cause of the bug.
If there are no branches between the target statement and the closest extent of the proximal control trace, the code is instrumented by traceCheckpoint nerves on each statement between the proximal control trace and the target statement. The last traceCheckpoint nerve that is executed points to the statement that caused the bug.
If the selected procedure has not even been called, there are two subcases: the procedure is xe2x80x9cpublicxe2x80x9d (available to be called externally); or private (not available to be called externally). Each statement that references the target procedure is instrumented. The list is presented for refinement to the developer. Each selection recursively generates a new xe2x80x9cfind why not calledxe2x80x9d bug.
If the selected procedure is public, the code may simply not be called eternally Private calls can be instrumented, but this is not the limitation of the invention""s capabilities in this area.
After the invention finds the proximal control trace, it suggests the source of the bug to the developer by displaying the expression along with the current value(s) is presented to the developer along with the choice to change the Boolean value to force the branch to be executed.
The invention then presents a visual representation of the branches along with its hypothesis that incorrect decisions were made at the intervening branch(es). The developer is then invited to edit the branch along with some suggested code. The invention""s formation of hypotheses differs from prior art in that it is based on actual data from running program as opposed to syntactic analysis or simulation.
Automated testing permits the invention to not only form but also test hypotheses. The tests available to the developer include, but are not limited to, Implicit Ul tests. The invention integrates tests from its own Ul tests in the data store. The invention stores hierarchies of tests such that they can be executed in their entirety or as sub-nodes. The tests are also extensible to include external tests provided they return a Boolean value.
Since each bug has a related test, the invention forms testable hypotheses about what the underlying cause of the bug is. If a developer takes one of the suggestions, the invention makes the edit and then reruns the test. If the statement is executed, the bug is marked fixed. This automated detect, analyze, suggest/edit, retest cycle greatly reduces the time to find and fix bugs because the only human intervention is to review the analysis and edit.
Find the origin of bad data motivates the fourth type of bug. The invention extends the mechanism used in control flow bugs by adding a target expression. The tracing mechanism of the nerves is used to record the values of local, class, and global variables that contribute to the composition of the target expression. The values of the constituent expression and the target expression constitute the xe2x80x9cproximal data treexe2x80x9d.
The developer selects a target statement, which generates an incorrect value (xe2x80x9ctarget expressionxe2x80x9d), and specifies a condition that the expression should be equal, for example, xe2x80x9cx less than   greater than 5xe2x80x9d would be what the developer types when the variable xe2x80x9cxxe2x80x9d is unexpectedly equal to 5. The target expression becomes the root of the proximal data tree.
The invention then parses the procedure containing the target expression, finding statements that reference symbols comprising the target statement. These statements are added to the proximal data tree as descendents of the root node. Previous teachings have shown the limitations of syntactic analysis; so, the invention uses run-time information to fully populate the proximal data tree.
The invention reviews the symbols in the tree and sets watch points so that the run-time values will be reported to the monitor process. When the target statement is executed, the monitor process stops target code execution and populates the proximal data tree, which now contains nodes with run-time values.
The proximal data tree is presented to the developer who then examines the values to identify, the cause of the data bug. The developer then goes to the statement generating the incorrect value and edits, and as with other automated bugs, the invention will retest and check the fix.
If the developer does not have sufficient information to attempt to resolve the bug, he selects nodes and the test is reexecuted; thus, progressively expanding the data node until the source of the bug is found. As each edit is made, the test is reexecuted, and the data tree is updated until the target expression is correct. The data bug is then marked closed.
As with the other automatic bugs, the progress of data bugs is reflected in the modified planning document.
The invention marks certain tests in the data store as regression tests. A test becomes a regression test if it is marked as such by either the developer or manager. Regression testing allows the manager and developer to see if old problems reemerge as the target code progresses. These tests are designated by the developer or the manager as crucial to the success of the project. It is easy to find and read reports of which tests are and are not passing. When all the regression tests are marked as consistently passing, the project is nearing completion.
Regression tests motivated a special type of automatic bug, xe2x80x9ca revert until broken bug.xe2x80x9d This type of bug occurs when edits break regression tests. A revert until broken bug is formed when the developer selects a test. The test is run on the current version. If the test fails, the invention xe2x80x9crevertsxe2x80x9d the target code to the version that the test passed. If this version passes, the invention goes back to the current version and then steps back version by version until a passing version is found. The version created after the last working version is the version that broke the regression test. The invention xe2x80x9cdiffsxe2x80x9d the working and non-working versions of the target code. This xe2x80x9cdiffxe2x80x9d identifies the particular sets of edits that broke the test.
Previous teachings have automated the correlation of versions with test status; however, these versions had to be explicitly xe2x80x9cchecked-inxe2x80x9d by the developer. Between check-ins, the target code may be edited several hundred times. What the invention teaches is the proactively storing of versions on every edit is necessary to localize the particular edits that broke the test.
Regression tests can take a long time to run. The exemplary embodiment provides a mechanism for scheduling and running such jobs during system idle time when the hardware is not actively interacting with a user. The contents and priorities of this queue of idle time tasks, referred to as night jobs, are alterable by the developer.
There are three classes of night jobs currently recognized by the exemplary embodiment.
Coverage analysis: determining which tests call which procedures. Coverage is a straightforward extension of nerve insertion. TraceEnter and traceExit nerves are inserted in all procedures. A test is run and the list of procedures is written to the data store. Coverage is performed by either 1) the developer""s choosing to activate it during normal target code execution, or 2) choosing a to run a test as a night job with coverage on.
Regression testing: if a test as marked as a xe2x80x9cregression testxe2x80x9d, it is automatically added to the night job (Automatic addition is however a user selectable option.) If a bug is marked as fixed and a procedure in its coverage path is edited since the last time the test was run successfully, this test is also automatically added as a night job. If a test breaks, the invention automatically creates revert until broken bugs and works them as described earlier.
External tasks: the invention also permits the user to use night jobs to coordinate other tasks that are external to the invention by providing a mechanism for running batch files.
The list of pending night jobs is kept in the data store. After the invention determines that the host computer is idle, all night jobs are placed in a common queue and executed in priority order until their queue is exhausted or until the running of the night jobs is interrupted by the developer. If a night job is interrupted by the developer, the job being executed is marked as interrupted. The results of night jobs are displayable by the developer such that they can be easily rescheduled. The results of night jobs, such as newly broken regression tests, are visible through the Gantt chart.
The invention provides a combined bug list that allows the integration of traditional, text-only xe2x80x9cnonautomaticxe2x80x9d bugs. Text only bugs can be linked by the developer to automatic bug(s), thus providing another link between the planning document and code. The combined list can include entries from other electronic bug lists. Further, a manager can make modifications in the combined bug list, which provides another avenue. For two-way communication between developer and manager.
Developers are commonly assigned to finish, alter, fix, or otherwise work on code written by another person. Unfortunately, taking over work on someone else""s code is a difficult proposition at best. Most developers keep much of the state of the code in their heads,even if the previous developer is available for questioning, communicating this information is frequently frustrating. The invention does not solve this problem, but it can lend assistance in a number of areas, making the xe2x80x9cnew developer""s xe2x80x9d task of adopting a xe2x80x9cprevious developer""s xe2x80x9d code easier.
The invention supports three types of transfer, which the invention meets each of these types of transfer with a different set of mechanisms.
The new developer may be asked to evaluate the state of the previous developer""s code. If the new developer has been asked to evaluate the target code, the invention presents test results, regression tests, unresolved crash bugs. The invention obtains a sorted list of procedures, where the procedures are sorted by to present the most recently modified procedures first. The invention presents other filters on this procedure list including procedure length, involvement in bugs, and the number of times a procedure is called during debugging or coverage.
The new developer may be asked to continue where the previous developer stopped. If the new developer is tasked to extend the target code, the invention shows the statements and procedures where the previous developer was working. If the new developer is tasked to extend the capabilities of the target code, the new developer uses the invention to create regression tests by turning on Ul tests while becoming aware of the program capabilities. With the capability, the invention""s revert until broken feature allows the new developer freedom to edit with the confidence that inadvertently broken code will be identified early. This information provides a dynamic picture of the code and provides starting points for further investigation.
The new developer may be asked to correct or modify undesirable behavior in the previous developer""s codexe2x80x94usually bugs. If the new developer is tasked with debugging, the invention presents the new developer with the combined bug list. This list also has facilities for sorting and filtering these bugs by priority, date created, type, and procedure. The invention looks for regression tests associated with those flawed behaviors.
Finally, as the data store""s schema is available, it is also transferable to other data stores which are compatible with other applications, providing more extensive access to the information collected by the invention.